Work for saxophonist, 3D animation and real-time sound processing controlled by motion capture data

We address issues raised in designing a new work for saxophonist and digital media involving 3D motion capture technology. A saxophonist’s kinaesthetics are recorded with a 3D motion capture system, and used as control data in the real-time processing of sound and 3D visuals. We address current gesture research and propose an extension to current gesture taxonomies. Linguistics have been taken as a basis for analysis of communication in musical processes, and serve as a fundamental reference in our discussion. The writings of Julia Kristeva, in particular her concept of dialectic oscillation between the semiotic and symbolic modalities, inform our approach to designing a system which articulate gestural intention with technological imprint. Notions of physicality and effort in the context of sound based performance prove useful in this research as the sophistication in processing that is associated with real-time audio-visual systems is often not complemented by gestural intention. By focusing on short performance segments we are able to test various mapping strategies that address the issue of relating control-spaces of different dimensions and qualities. The resulting connections between the performer’s gesture (as recorded by the motion capture system) and the sonic output are a product of the character of each individual segment and its context within the work. The work is designed using the MAX/MSP/Jitter graphic-programming environment.